1. Field of the Invention
The present invention relates to a method for automatic software testing in general, and to a method and apparatus for automatically generating and performing tests associated with an application, in particular.
2. Discussion of the Related Art
Testing, or quality assurance of applications, is an inseparable part of the development process. Software testing is required to test all the possible uses of an application, on all types and combinations of data or environments, whether they are of high probability to occur or not. The diligence is required to ensure coverage of the product, i.e. discovering and possibly repairing as many as possible problems and failures of the application, as early as possible. Best practiced quality assurance should accompany a product along all its life cycle phases, including design, development, pre deployment, post deployment and as a customer service tool for reconstructing customer-reported problems. Different tests might be planned for each phase, in order to cover the different aspects and functionality that have to be tested at that phase. However, the time and manpower resources dedicated to testing are not always sufficient, which may cause suboptimal quality of the products. In addition, the testing is sometimes monotonous and tedious, thus causing a human quality tester to omit, change or otherwise perform tests in an inconsistent manner. Another problem results from long execution times of tests, which result in lack of agility when the application or the testing has to be changed.
Automatic testing tries to overcome these problems by assuring that all tests are performed as planned, thus improving the coverage and quality as well as saving manpower resources. Currently existing systems record executions of an application, storing inputs and actual results, or receive the description of the tests as designed or performed by a human and later rerun the tests and expect to receive the same results. However, automatic testing suffers from a number of drawbacks. A substantive development effort has to be put into generating a thorough test script that will cover all areas and all functionality of the application. In addition, test scripts are sensitive to changes in the application. Even minor changes such as changing the text or the location of a control cause most test scripts not to recognize a previous test and thus are unable to perform the test. In such cases, changing and regenerating the test script is required for adapting the tests to the modified application. Yet another drawback relates to the test being linked to a specific run-time platform or tool, such as Mercury QTP manufactured by Mercury (http://www.mercury.com), or to a specific runtime environment, such as Flash, HTML, Adobe Flex or others. Porting a test script to another tool or another environment requires effort which is similar to the initial generation. An additional drawback relates to a test being coupled to a specific test data, which generally does not enable customer specific settings, which in turn harms the thoroughness of the test and hence the product's quality. Yet another drawback of these methods results from the tests being planned and designed by human beings, and thus being vulnerable to possible incompleteness, meaning that the collection of tests might not cover all possible activities in the system, and miss possible failure points.
There is therefore a need for a method and apparatus for test generation and execution for applications. The method and apparatus should be independent of the used testing platforms and runtime environments, and the designed tests should be easily adapted to changes in an application, without requiring manual changing by a user.